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One  of  the  most  popular  approaches  to  satisfying  this  feedback  requirement  is  to 
videotape  the  student  in  interpersonal  situations.  Most  such  efforts,  however, 
fall  quite  short  of  providing  appropriate  feedback  and  the  general  failure  to 
demonstrate  training  benefits  from  the  use  of  this  method  can  be  attributed  to 
this  shortcoming.  Further,  the  costs  and  difficulties  associated  with  providing 
adequate  feedback  regarding  interpersonal  performance  apparently  have  discour- 
aged attempts  to  do  so. 

This  report  presents  an  approach  for  decreasing  some  of  these  costs  and  diffi- 
culties. This  approach  uses  a computer  to  assist  in  the  process  of  eliciting 
appropriate  feedback  from  judges  and  providing  this  feedback  to  the  student, 
both  in  coordination  with  the  videotaped  recordings  of  the  student's  performance 
In  this  regard,  computer  programs  designed  to  help  a judge  rate  interpersonal 
performance  along  a variety  of  dimensions  were  developed  and  tested.  Although 
these  programs  also  are  suitable  for  providing  feedback  to  the  student,  formal 
tests  of  this  aspect  of  the  process  were  not  conducted  in  the  present  research. 
Formal,  albeit  brief .^experimental  evaluations  of  the  feedback  elicitation  and 
development  programs  were  conducted,  however. 

Several  potential  benefits  of  the  procedures  for  interpersonal  skills  training 
were  demonstrated  as  a result  of  both  the  formal  and  informal  tests  of  the 
system. c 
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NAVTRAEQU IPCEN  IH-308 
PREFACE 
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NAVTRAEQU IPCEN  IH-308 
SUMMARY 


Training  technology  in  the  interpersonal  skills  area  has  not  kept  pace 
with  the  training  technology  that  is  available  in  most  other  skill  areas.  The 
deficiencies  in  interpersonal  skills  training  are  especially  apparent  in  "job 
oriented"  instruction,  wherein  the  student  performs  operational  tasks  in  simu- 
lated or  actual  job  situations.  A major  obstacle  to  the  application  of  job 
oriented  instruction  to  this  area  is  the  difficulty  in  providing  appropriate 
feedback  to  the  student  concerning  his  performance  in  the  interpersonal  situa- 
tion. 


One  of  the  most  popular  approaches  to  satisfying  this  feedback  requirement 
is  to  videotape  the  student  in  interpersonal  situations.  Most  such  efforts, 
however,  fall  quite  short  of  providing  appropriate  feedback  and  the  general 
failure  to  demonstrate  training  benefits  from  the  use  of  this  method  can  be 
attributed  to  this  shortcoming.  Further,  the  costs  and  difficulties  associated 
with  providing  adequate  feedback  regarding  interpersonal  performance  apparently 
have  discouraged  attempts  to  do  so. 

This  report  presents  an  approach  for  decreasing  some  of  these  costs  and 
difficulties.  This  approach  uses  a computer  to  assist  in  the  process  of 
eliciting  appropriate  feedback  from  judges  and  providing  this  feedback  to  the 
student,  both  in  coordination  with  the  videotaped  recordings  of  the  student's 
performance.  In  this  regard,  computer  programs  designed  to  help  a judge  rate 
interpersonal  performance  along  a variety  of  dimensions  were  developed  and 
tested.  Although  these  programs  also  are  suitable  for  providing  feedback  to 
the  student,  formal  tests  of  this  aspect  of  the  process  were  not  conducted  in 
the  present  research.  Formal,  albeit  brief,  experimental  evaluations  of  the 
feedback  elicitation  and  development  programs  were  conducted,  however. 

Several  potential  benefits  of  the  procedures  for  interpersonal  skills 
training  were  demonstrated  as  a result  of  both  the  formal  and  informal  tests 
of  the  system.  First,  the  tasks  of  eliciting,  developing,  and  delivering  feed- 
back appeared  to  be  facilitated  by  the  programs.  Evidence  for  this  came  mainly 
from  the  experiences  of  project  personnel  who  noted,  in  developmental  tests  of 
the  system,  that  these  training  tasks  were  difficult,  frustrating,  and  some- 
times even  impossible  to  perform  correctly  without  computer  assistance.  Also, 
in  the  formal  evaluations  of  the  computer-assisted  methods,  judges  reported 
that  the  work  load  was  "about  right,"  that  the  task  was  interesting,  and  that 
they  understood  the  task. 

Second,  the  feedback  that  was  made  possible  by  this  new  technique  appeared 
to  be  much  more  useful  than  the  feedback  that  typically  results  from  other  ap- 
proaches in  the  area.  This  conclusion  was  based  on  an  analytical  comparison 
of  the  extent  to  which  the  feedback  of  the  new  programs  vs.  that  of  other,  non- 
computer-aided methods,  conformed  to  the  characteristics  of  good  feedback. 

Third,  the  computer  programs  can  be  used  to  improve  the  reliability  of 
ratings  of  interpersonal  performance  and,  in  so  doing,  improve  the  validity  of 
the  ratings.  Although,  for  a variety  of  possible  reasons,  overall  rating 
reliabilities  were  relatively  low,  a new  computer-aided  technique  improved  the 


3 


NAVTRAEQUIPCF.N  IH-308 


J 


t*e1  i abi  1 i ty  on  one  of  the  two  variables  (considerate)  to  which  It  was  applied. 
This  technique  provided  computer-generated,  t;me-weighted  averages  of  a series 
of  individual  ratings  of  a variable.  The  reliability  resulting  from  this 
method  was  considerably  greater  than  a global  rating  made  by  the  judge  on  the 
variable. 

The  preliminary  nature  of  this  study  prevented  extensive  refinement  of 
the  programs  and  the  procedure  for  their  use.  For  example,  the  judges  were 
not  extensively  trained  in  the  use  of  the  computer  rating  methods,  and  little 
was  done  to  assure  that  they  performed  conscientiously.  Also,  more  should  be 
done  to  assure  that  the  functions  and  formats  of  the  programs  are  appropriate. 
These  latter  Improvements  await  more  extensive  development  and  research  ef- 
forts. These  efforts  should  result  in  a more  complete  program  of  computer 
aids,  after  which  a more  extensive  effort  to  apply  and  evaluate  the  programs 
should  be  conducted. 

The  present  study  demonstrated  the  probability  that  a variety  of  computer 
programs  can  enhance  interpersonal  skills  training.  The  nature  of  ideal  com- 
puter programs  for  this  purpose  is  a subject  for  further  research. 

Further  research  is  also  needed  to  determine  whether  students  perform  any 
differently  (hopefully  better)  as  a result  of  this  new  feedback  procedure,  and 
how  these  changes  compare  to  those  occurring  under  more  traditional  (i.e.,  non- 
computer) training  methods.  This  training  effectiveness  issue  (i.e.,  how  well 
does  the  training  change  performance)  should  be  addressed  after  additional 
development  and  refinement  of  the  new  technique.  The  concern  about  training 
validity  (I.e.,  how  well  does  the  training  improve  performance)  is  a much  more 
thorny  problem,  since  universal  agreement  on  the  characteristics  and  measure- 
ment of  good  leadership  does  not  exist,  despite  a great  body  of  research  on 
the  subject.  The  new  technology  presented  here  is  intended  to  be  content  free. 
That  is,  it  should  teach  the  subject  matter  of  interpersonal  instruction  more 
efficiently,  regardless  of  the  nature  or  quality  of  this  subject  matter.  More 
efficient  training  strategies  are  necessary,  but  not  sufficient,  conditions  to 
improve  interpersonal  performance.  Improved  performance  in  this  area  also 
requires  valid  subject  matter,  and  this  concern  was  only  ancillary  to  the 
goals  of  the  present  effort. 

Finally,  once  the  training  effectiveness  and  validity  issues  have  been 
resolved,  cost-effectiveness  issues  arise.  Cost-effectiveness  questions  re- 
quire comparisons  of  the  new  technique  with  other  logical  approaches  to  the 
objectives.  It  was  beyond  the  scope  of  this  study  to  develop  and  evaluate 
alternate  approaches.  For  this  reason,  and  because  the  training  effectiveness 
and  validity  issues  have  yet  to  be  resolved  for  the  new  technique,  rigorous 
efforts  to  answer  cost-effectiveness  questions  are  premature.  Nevertheless, 
assuming  the  desirability  of  the  functions  of  the  new  computer  techniques,  and 
comparing  these  techniques  with  purely  manual  operations  for  performing  the 
same  functions,  considerable  cost  savings  can  be  demonstrated. 
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NAVTRAEQU IPC EN  IH-30R 
SECTION  I 
INTRODUCTION 

The  efficiency  of  an  organization  depends  on  the  effectiveness  of  inter- 
personal cormunications  within  it.  Typically,  little  is  done  to  improve  per- 
formance in  this  area. 

The  Naval  Training  Equipment  Center  (NAVTRAEQU IPCEN) , during  the  past  five 
years,  has  been  engaged  in  determining,  developing,  and  evaluating  several  ap- 
proaches to  the  improvement  of  interpersonal  behaviors  through  the  use  of  a 
computer.  These  approaches  have  mainly  been  aimed  at  improving  the  inter- 
personal communications  abilities  of  Navy  recruit  company  cormanders  (CCs)  in 
interactions  with  their  recruits.  These  earlier  approaches  are  described  in 

other  reports.1 ,?’3’4,5’6’7, 8,9 

One  of  these  approaches  involves  the  use  of  videotapes  to  teach  inter- 
personal skills.  The  computer  controls  the  operation  of  a videotape  player, 
coordinating  the  tape  with  written  material  presented  on  a CRT  terminal.  The 
tapes  consist  of  both  real  and  role-played  interactions  between  CCs  and  re- 
cruits. The  lessons  train  and  test  the  ability  to  recognize  examples  of  vari- 
ous interpersonal  skills.  These  videotape  lessons  follow  an  extensive  set  of 
written  CAI  lessons  which  cover  the  same  skills.  The  video  lessons  were 
designed  to  present  the  skills  in  a form  which  more  closely  approximates  the 
real  operational  setting. 


Y.  Spencer,  G.  J.  and  Hausser,  D.  L.;  Blaiwes,  A.  S.  and  Weller,  D.  R.  Use  of 
Computer-Assisted  Instruction  for  Interpersonal  Skill  Training  - A Pilot  Study, 

1975.  Technical  Report:  NAVTRAEQU IPCEN  73-C-01 33-1 . 

2.  Cohen,  J.  L.  and  Fishbein,  M.  Development  and  Research  Utilizing  the 
PLATO  IV  System  for  Company  Commander  Behavioral  Change  Training.  August  1975. 
Technical  Report:  NAVTRAEQU IPCEN  73-C-01 29-1 . 

3.  Hausser,  D.  L.,  Blaiwes,  A.  S.,  Weller,  D.  R.,  and  Spencer,  0.  J.  Applica- 
tion of  Computer-Assisted  Instruction  to  Interpersonal  Skill  Training.  January 

1976.  Technical  Report:  NAVTRAEQU IPCEN  74-C-Ol  00-1 . 

4.  Cohen,  J.  L.  and  Fishbein,  M.  A Field  Test  of  the  PLATO  IV  System  for 
Company  Commander  Behavioral  Change  Training.  July  1976.  Technical  Report: 
NAVTRAEQU IPCEN  74-C-0095-1. 

5.  Lukas,  G. , Blaiwes,  A.  S. , and  Weller,  D.  R.  Evaluation  of  Human  Relations 
Training  Programs.  January  1977.  Technical  Report:  NAVTRAEQU IPCEN 
75-C-0076-1 . 

6.  Blaiwes,  A.  S.,  Weller,  D.  R. , and  Romot,  G.  Development  and  Implementa- 
tion of  a Computerized  Evaluation  and  Training  System  (CETS)  at  a Recruit 
Training  Command.  March  1978.  Technical  Report:  NAVTRAEQU IPCEN  IH-300. 

7.  Weller,  D.  R.  and  Blaiwes,  A.  S.  Leadership  Dimensions  of  Navy  Recruit 
Company  Commanders  and  Recruit  Morale  and  Performance.  Psychological  Reports, 
1976,  39,  767-770. 

8.  Blaiwes,  A.  S.  and  Weller,  D.  R.  A Social  Simulator:  Development  and 
Evaluation.  Educational  Technology.  March  1977,  14-20. 

9.  Blaiwes,  A.  S.  and  Weller,  D.  R.  A Computerized  Evaluation  and  Training 
System  (CETS)  for  Recruit  Training  Commands:  An  Overview.  NAVTRAEQU IPCEN,  in 
preparation. 
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The  current  research  is  directed  at  enlarging  the  use  of  videotape  in  the 
training  of  interpersonal  skills.  An  idea1  videotape  training  program  would 
start  with  the  taping  of  a student  in  an  actual  job  situation.  The  tape's 
contents  would  then  be  rated  for  the  performance  of  the  interpersonal  skills. 
Then  the  student  would  be  given  feedback  on  his/her  performance,  with  special 
attention  to  strong  and  weak  areas.  Additional  training  would  then  be  given 
on  appropriate  skills.  The  taping  - feedback  - training  cycle  would  be  re- 
peated until  performance  was  acceptable. 

A critical  step  in  the  development  of  this  program  is  the  design  of  a 
method  for  accurately,  efficiently,  and  reliably  rating  interpersonal  behavior 
as  seen  on  videotape.  In  the  effort  to  develop  a better  method  for  rating 
interpersonal  behavior,  many  aspects  of  the  feedback  component  also  were  pro- 
vided. These  are  the  subjects  of  this  study. 

It  was  decided  at  the  outset  that  a computer  would  be  used  to  assist  in 
the  rating.  Some  of  the  more  apparent  advantages  of  using  a computer  are: 

a.  Raters'  responses  can  be  automatically  keyed  to  the  location  on  the 
tape  where  they  occur. 

b.  Raters'  responses  can  be  automatically  stored  and  processed. 

c.  Complex  outputs  and  displays  can  be  quickly  produced. 

d.  All  functions  of  the  tape  player  can  be  under  programmed  control. 

e.  The  most  effective  approaches  to  videotaped  rating  can  be  standardized. 

The  need  for  research  and  development  ( R&D ) on  the  process  of  rating, 
analyzing,  and  providing  feedback  on  a student's  videotaped  performance  is 
indicated  by  the  widespread  practice  of  using  videotapes  for  performance  feed- 
back in  training  programs,  contrasted  with  the  relatively  minor  experimental 
evidence  which  would  recommend  their  use.  This  is  especially  true  in  the 
interpersonal  skills  area. 

Most  evidence  in  favor  of  such  use  of  videotapes  appears  to  be  non-quanti- 

tative  and  anecdotal.  For  example,  Kanner^  reports  that  instructors  become 
vividly  aware  of  their  shortcomings  as  a result  of  viewing  their  performance 
on  videotape  which  "...is  often  followed  by  striking  changes  in  the  instruc- 
tor's teaching  behavior....  These  insights  are  often  achieved  much  more 
rapidly  than  when  someone  else,  without  the  benefit  of  videotape  playback, 
criticizes  an  instructor's  performance."  A similar  observation  was  made  in  an 
effort  to  compare  videotape  and  role-playing  procedures  for  teaching  job  inter- 
view skills.^  It  was  noted  "...that  several  subjects  in  the  role-playing 


THT  Kanner,  J.  H.  In  a Report  of  the  Third  Armed  Forces  Television  Confer- 
ence. Fort  Lee,  VA,  17-19  October  1962.  Office  of  The  Chief  Signal  Officer 
Audio-Visual  Communications  Directorate,  April  1963,  p.  14. 

11.  Venardos,  M.  G.  and  Harris,  M.  B.  Job  Interview  Training  with  Rehabilita- 
tion Clients:  A Comparison  of  Videotape  and  Role-Playing  Procedures.  Journal 
of  Appl ied  Psychology,  1973,  vol.  58,  No.  3,  365-367. 
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group  initially  resisted  the  possibility  of  their  having  interview  problems, 
whereas  those  in  the  videotape  group  were  quick  to  detect  their  own  Inappro- 
priate behaviors  and  acknowledge  the  need  for  improvement." 

More  quantified  observations,  however,  do  not  seem  to  recommend  the  use 
of  videotapes  so  strongly.  For  example,  videotaped  performance  feedback  did 
not  Improve  group  performance  in  a problem  solving  situation  when  used  alone; 
although  it  did  add  small  improvements  when  combined  with  a videotaped  model 
1 2 

of  performance.  Similar  results  were  obtained  in  an  experiment  which  tested 
the  effects  of  model  and  feedback  videotapes  on  teachers'  use  of  positive  feed- 

1 3 

back  responses  to  children's  behavior.  "The  findings  demonstrated  that  the 
combined  model  and  feedback  videotape  technique  and  the  model  videotape  tech- 
nique are  effective  while  the  feedback  videotape  intervention  is  ineffective 
in  changing  teaching  behaviors." 

Other  researchers,  noting  the  cost  of  videotape  procedures,  recommend 
other  methods  to  achieve  the  training  goals.  For  example,  when  no  difference 
was  found  between  role-playing  and  videotape  procedures  for  the  job  interview 
training  (both  being  superior  to  an  attention-placebo  control),  the  authors 
concluded,  with  some  reservation,  that:  "While  videotape  is  being  used  in- 
creasingly today  for  teaching  various  interpersonal  skills,  such  as  inter- 
viewing, the  savings  in  equipment  expense  that  would  result  from  employing  the 
equally  effective  procedure  of  role-playing  might  suggest  that  role-playing  be 

1 4 

the  treatment  of  choice." 

Another  effort  to  supplement  operational  procedures  for  training  instruc- 
tors at  Sheppard  Air  Force  Base  with  videotaped  performance  feedback  resulted 
in  conclusions  as  follows:  "That  those  capabilities  have  potential  payoffs  is 
assumed,  but  it  is  not  clear  how  those  capabilities  can  properly  be  applied.... 
Use  of  the  system  as  a supervisory  tool  in  the  existing  school  at  Sheppard  Air 

Force  Base  offers  no  net  advantages.  The  payoffs  do  not  warrant  the  cost."^ 

Some  key  words  in  this  latter  quotation  are  "...how  . . . (the  performance 
feedback  videotapes)  can  be  properly  applied."  Little  seems  to  be  done  to  im- 
prove the  quality  of  the  videotaped  experience  beyond  merely  allowing  the  stu- 
dent to  view  the  tape,  and  even  less  seems  to  be  known  about  what  can  be  done 
to  enhance  the  effectiveness  of  such  training. 


TT.  Walter,  G.  A.  Effects  of  Training  Inputs  on  Group  Performance.  Journal 
of  Applied  Psychology,  1975,  vol.  60,  No.  3,  308-312. 

13.  Rutherford,  R.  B.  The  Effects  of  a Model  Videotape  and  Feedback  Video- 
tapes on  the  Teaching  Styles  of  Teachers  in  Training.  The  Journal  of  Experi- 
mental Education.  Fall  1973,  vol.  42,  No.  1,  64-69. 

14.  See  footnote  10  on  page  8. 

15.  Hays,  J.  and  Pulliam,  R.  Development  and  Evaluation  of  Video  Systems  for 
Performance  Testing  and  Student  Monitoring.  Technical  Training  Division, 
Lowery  Air  Force  Base,  CO,  AFHRL-TR-74-67 , July  1974,  p.  169. 
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One  significant  exception  to  this  statement  is  a research  program  on  the 
"interpersonal  process  recall  technique"  applied  to  counseling  guidance 
training.  This  approach  "...consists  of  recording  a counselor  trainee  and 
client  session  followed  by  an  interrupted  playback  for  the  participants  in 
Separate  rooms.  During  the  interrupted  playback,  the  participants  are  encour- 
aged to  recall  thoughts  and  emotions  by  specially  trained  interrogators; 

interrogation  sessions  are  audiorecorded."  ’^7 


Some  more  restricted  techniques  for  "properly  applying"  the  performance 
feedback  videotapes  include:  (a)  using  orientating  comments  for  focusing  stu- 

1 8 

dents'  perceptions  during  videotaped-feedback  change  attempts,  and  (b)  urging 
students  to  corment  on  their  own  performance  and  the  performance  of  others  in 
19 

a group. 


The  thrust  of  the  present  research  is  to  extend  our  knowledge  of  such 
techniques  and  our  capabilities  for  using  them  for  the  enhancement  of  video- 
taped performance  feedback  in  interpersonal  training  applications,  and  espe- 
cially to  determine  the  contributions  to  be  made  by  a computer  to  these 
processes. 


' 


4 


Tin  Murmin , J.  ft  Report  on  the  Fifth  Armed  Forces  Television  Conference, 
Sheraton-Park  Hotel,  Washington,  DC,  2-3  November  1964.  Office  of  the  Chief 
of  Communications-Electronics  Pictorial  and  Audio-Visual  Directorate,  June 
1965,  p.  49. 

17.  Kagan,  N.  Can  Technology  Help  Us  Toward  Reliability  in  Influencing  Human 
Interaction?  Educational  Technology,  Feb.  1973,  44-50. 

18.  See  footnote  11  on  page  8. 

19.  See  footnote  10  on  page  8. 
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SECTION  II 


GENERAL  METHODOLOGY 

Three  important  factors  in  the  design  of  an  interpersonal  skills  rating 
method  are:  (1)  reliability,  (2)  ease  and  convenience  of  rating,  (3)  useful- 
ness of  resulting  ratings.  The  meanings  of  the  first  two  items  are  fairly  ob- 
vious, but  the  third  requires  some  elaboration.  The  principle  purpose  of  an 
interpersonal  skills  rating  of  CCs  is  to  provide  them  with  feedback  on  their 
performance  which  will  lead  to  an  improvement  of  their  performance.  Therefore, 
the  results  of  the  rating  procedure  must  be  easily  understood  by  the  CCs,  and 
specific  enough  so  that  the  CCs  will  know  what  steps  to  take  to  improve  their 
performance.  Also,  the  feedback  must  be  valid,  such  that  a CC's  performance 
will  in  fact  improve  as  a result  of  taking  the  steps  suggested  by  the  feedback. 
A second  purpose  of  the  ratings  is  to  provide  information  to  management  for 
use  in  decision  making  and  organizational  development.  Thus,  the  outputs  of 
the  rating  procedure  must  be  suitable  for  this  purpose  also. 

It  was  decided  that  a two-step  process  would  be  used  to  design  and  test 
the  interpersonal  skills  rating  method.  In  the  first  step,  three  different 
rating  methods  were  devised.  These  methods  were  then  tested,  using  a small 
group  of  raters.  The  three  factors  mentioned  above  were  all  used  in  evaluating 
these  methods.  In  the  second  step,  a new  rating  method  was  devised,  incorpor- 
ating the  best  features  of  the  preliminary  methods.  A small  group  of  raters 
also  provided  information  for  evaluating  this  method. 

RATING  VARIABLES 

The  variables  to  be  rated  were  selected  from  those  isolated  during  earlier 
phases  of  the  research.  Also  considered  were  performance  variables  employed  in 
a variety  of  other  interpersonal  training  and  evaluation  systems  (e.g., 

PQ  2"|  22 

Klauss,  Fleishman,  and  Bowers  ).  The  particular  variables  employed  here 
were  selected  because  of  their  commonality  to  many  of  these  systems  and  because 
they  appeared  in  pilot  tests  to  be  more  amenable  than  others  to  the  present 
rating  context.  There  were  three  levels  of  variables  chosen:  global,  skill, 
and  behavior. 

GLOBAL  LEVEL.  At  this  level,  the  raters  were  to  judge  the  overall  effective- 
ness of  the  CC  (how  well  the  CC  was  meeting  the  goals  of  the  interaction). 

SKILL  LEVEL.  At  this  level,  the  raters  were  to  judge  the  CC's  performance  in 
two  skill  areas: 


TUI  Klauss,  llucH.  Measuring  the  Impact  on  Subordinates  of  Managers'  Inter- 
personal Communication  Styles  and  Credibility.  The  Maxwell  School  of  Citizen- 
ship and  Public  Affairs,  Syracuse  University,  TR-2,  1977. 

21.  Fleishman,  E.  A.  Twenty  Years  of  Consideration  and  Structure  in  E.  A. 
Fleishman  and  J.  G.  Hunt  (Eds.)  Current  Developments  in  the  Study  of  Leader- 
ship. Carbondale:  Southern  Illinois  University  Press,  1973. 

22.  Bowers,  D.  G.  and  Seashore,  S.  E.  Predicting  Organizational  Effectiveness 
with  a Four-Factor  Theory  of  Leadership.  Administrative  Science  Quarterly, 

Vol . II,  No.  2,  Sept.  1966,  pp  238-263. 
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a.  Considerate  - To  what  extent  does  the  CC  treat  the  recruit  like  a 
human  being? 

b.  Clear  - How  clear  is  the  CC's  conmunication  with  the  recruit? 

BEHAVIOR  LEVEL.  At  this  level,  the  raters  were  to  judge  the  CC's  performance 
of  certain  behaviors  (both  positive  and  negative).  The  sixteen  behaviors 
selected  fall  under  positive  and  negative  representations  of  the  two  skill 
areas: 


a.  Considerate  - Positive 

(1)  The  CC  asked  the  recruit  how  he/she  felt  about  the  situation. 

(2)  The  CC  expressed  confidence  in  the  recruit's  abilities. 

(3)  The  CC  was  patient  with  the  recruit. 

(4)  The  CC  used  the  recruit's  name. 

b.  Considerate  - Negative 

(1)  The  CC  showed  disinterest  in  the  recruit's  feelings. 

(2)  The  CC  belittled  the  recruit's  abilities. 

(3)  The  CC  was  impatient  with  the  recruit. 

(4)  The  CC  used  abusive  names. 

c.  Cl  ear  - Posi tive 

(1)  The  CC  gave  specific  details. 

(2)  The  CC  encouraged  recruits  to  ask  questions. 

(3)  The  CC  asked  questions  of  the  recruits  to  see  if  they  understood. 


(4)  The  CC  gave  reasons  to  show  the  relevance  of  what  he/she  was 

saying. 


Clear  - Negative 

(1 ) The  CC  was  vague. 

(2)  The  CC  discouraged  questions. 

(3)  The  CC  failed  to  question  the  recruits  for  their  understanding. 

(4)  The  CC  failed  to  give  reasons  although  the  situation  called  for 
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EXPERIMENTAL  APPARATUS 

This  experiment  used  a Nova  3/12  minicomputer,  a Sony  cassette  videotape 
player,  a Sony  T.V.  monitor,  a Tektronix  display  terminal,  and  a Data  General 
printer.  The  videotape  recorder  was  connected  to  the  computer  through  a 
specially  designed  interface  which  allowed  the  computer  to  control  all  of  the 
functions  of  the  player.  The  subjects  received  instructions  and  feedback  on 
the  terminal,  and  made  responses  on  the  terminal  keyboard.  The  printer  was 
used  for  data  output. 

Control  of  the  videotape  player  was  accomplished  through  timing.  For 
example,  it  was  necessary  to  determine  the  precise  spot  on  the  tape  which  was 
being  shown  when  a subject  made  a response,  so  that  the  tape  could  be  stopped 
automatically  at  the  same  spot  on  the  next  showing.  To  do  this,  the  computer 
kept  track  of  the  elapsed  time  into  the  segment  vtfien  each  response  was  made. 

On  subsequent  showings,  when  this  elapsed  time  was  reached,  the  computer  would 
automatically  activate  the  "pause"  mode  on  the  player,  causing  the  tape  to 
stop.  When  the  subject  completed  his  inputs  at  this  spot,  a terminal  keypress 
would  cause  the  computer  to  activate  the  "play"  mode,  and  the  tape  would 
continue. 
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PRELIMINARY  STUDY 

The  purpose  of  the  preliminary  study  was  to  determine  vrfiich  of  the  candi- 
date rating  methods  developed  for  this  study,  or  which  features  of  these 
methods,  were  to  be  retained  for  further  study. 

RATING  CONDITIONS 

A number  of  different  methods  for  rating  interpersonal  skills  with  com- 
puter assistance  were  conceived.  These  were  narrowed  down  to  three  candidate 
methods  which  utilized  those  features  which  seemed  to  be  most  promising  and 
which  could  be  implemented  within  the  resources  of  this  project.  Some  prom- 
ising features  were  not  included  at  this  time  due  to  limitations  in  the  experi- 
mental equipment.  For  example,  random  access  to  tape  segments  would  require 
signals  from  the  tape  player  to  the  computer,  and  at  present  the  signal  flow 
is  only  from  the  computer  to  the  tape  player.  The  reverse  is,  however,  tech- 
nically feasible.  Each  of  the  three  methods  yielded  scores  on  the  two  skills, 
but  different  procedures  were  used  to  arrive  at  these  scores.  Each  method  also 
yielded  scores  on  various  other  rating  variables. 

METHOD  A.  While  viewing  a videotape  of  an  interaction  between  a recruit  and  a 
CC,  the  rater  made  a continuous  rating  of  the  overall  effectiveness  of  the  CC. 
This  was  done  by  pressing  keys  1-5  on  the  computer  terminal  (1  = totally  in- 
effective, 5 = very  effective)  whenever  the  rater  noticed  a change  in  the  over- 
all effectiveness  of  the  CC's  performance.  The  computer  calculated  a time- 
weighted  mean  for  these  ratings  in  the  overall  effectiveness  category,  which 
comprised  the  "computer  overall"  score.  Following  this,  the  rater  viewed  the 
same  tape  segment  again.  This  time,  the  tape  stopped  at  each  point  where  the 
rater  changed  his/her  rating  during  the  first  viewing,  and  the  rater  was  asked 
to  report  whether  he/she  had  observed  the  CC  being  clear  or  considerate  during 
the  prior  segment.  The  rater  also  had  an  opportunity  to  type  in  a comment  at 
this  time.  Following  this  viewing,  the  computer  displayed  the  total  number  of 
times  each  skill  had  been  observed  and  the  rater  made  a summary  evaluation  for 
each  skill,  as  well  as  a summary  evaluation  of  overall  effectiveness  (providing 
the  "judge  clear",  "judge  considerate",  and  "judge  overall"  scores). 

METHOD  B.  While  viewing  a videotaped  interaction  between  a CC  and  a recruit, 
the  rater  made  a continuous  rating  of  the  CC's  performance  on  one  of  the  two 
skills  (by  pressing  keys  1-5).  The  computer  provided  time-weighted  means  for 
these  scores,  resulting  in  the  "computer  clear"  and  "computer  considerate" 
scores.  Following  this,  the  rater  viewed  the  same  segment  again.  This  time, 
the  tape  stopped  at  each  point  where  the  rater  changed  his/her  rating  during 
the  first  viewing,  and  the  rater  was  asked  to  select  the  behaviors  which  he/she 

had  seen  during  the  previous  segment  (from  the  eight  behaviors  under  the  skill 

in  question).  At  the  end  of  this  viewing,  the  total  number  of  observations  of 

each  behavior  was  displayed,  and  the  rater  made  a summary  rating  for  the  skill. 

Following  this,  the  entire  procedure  was  repeated  for  the  other  skill.  Final- 
ly, the  rater  made  an  overall  effectiveness  rating. 
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METHOD  C.  While  viewing  a videotaped  interaction  between  a CC  and  a recruit, 
the  rater  pressed  a key  on  the  termin’1  each  time  he/she  observed  one  of  the 
16  behaviors.  Following  this,  the  rater  viewed  the  same  segment  again.  This 
time,  the  tape  stopped  at  each  point  where  the  rater  had  marked  a behavior. 

The  rater  then  selected  the  behavior  which  he/she  had  seen.  Following  this 
viewing,  the  computer  displayed  the  total  number  of  observations  of  each  be- 
havior, and  the  rater  entered  a summary  score  for  each  skill.  Finally,  the 
rater  entered  an  overall  effectiveness  rating. 

The  actual  instructions  to  the  raters  are  in  Appendix  A. 

VIDEOTAPES 

Three  videotape  segments  were  selected  for  the  preliminary  study.  Each 
involved  a CC  interacting  with  a single  recruit.  One  of  the  segments  was  a 
real-life  interaction.  The  other  two  were  role-played  situations,  with  two 
CCs  playing  the  parts  of  CC  and  recruit.  The  role-playing  scenarios  are  in 
Appendix  B.  The  first  two  scenarios  are  the  ones  actually  used  in  the  study. 
Several  role-plays  were  videotaped,  and  the  two  most  appropriate  were  selected. 
Also  included  in  Appendix  B is  an  example  of  a behavioral  checklist,  which 
accompanies  the  second  role-play.  Such  checklists  were  used  in  CC  school 
classes  to  evaluate  role-players  and  stimulate  class  discussion.  The  check- 
lists were  provided  only  for  maximizing  the  benefits  of  the  role-plays  in  CC 
school  classes,  and  were  not  otherwise  involved  in  the  present  study.  Various 
personnel  at  the  Recruit  Training  Command,  Orlando  viewed  all  of  the  taped 
segments  and  verified  that  the  content  and  performances  were  realistic. 

PROCEDURE 

The  subjects  were  six  students  (two  female,  four  male)  from  Florida  Tech- 
nological University.  All  of  these  students  were  psychology  majors  with  some 
experience  with  behavioral  rating  techniques.  Each  subject  used  each  of  the 
three  rating  methods.  A given  tape  was  always  rated  by  the  same  method  by  all 
subjects  because  representing  all  method  X tape  segment  conditions  would  have 
resulted  in  only  two  subjects  per  condition,  which  would  be  insufficient  to 
compare  the  response  characteristics  (where  the  tape  is  stopped,  what  ratings 
are  given  at  these  points,  etc.)  of  different  subjects  under  similar  conditions. 

Following  each  method,  the  subjects  were  given  a questionnaire  on  their 
opinions  of  that  method.  The  questionnaire  is  reproduced  in  Appendix  C.  Each 
subject  was  also  interviewed  in  depth  by  the  experimenters. 

Outputs  from  the  computer  provided  a record  of  all  subject  responses.  In 
addition,  the  computer  provided  mean  ratings  for  methods  A and  B,  and  graphical 
displays  of  subject  responses  for  all  methods.  Sample  outputs  are  contained 
in  Appendix  D. 

RESULTS 

The  results  of  the  preliminary  study  are  organized  according  to  the  three 
factors  mentioned  earlier;  reliability,  ease  of  use,  and  usefulness  of  ratings. 
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RELIABILITY.  The  standard  deviation  of  each  score  was  used  as  ah  indication  of 
reliability.  The  smaller  the  standard  deviation,  the  greater  the  agreement 
across  judges,  and,  therefore,  the  greater  the  reliability.  The  standard 
deviations  are  presented  in  Table  1.  Method  C showed  the  highest  variability 
on  the  three  measures  which  were  common  to  all  methods  (judge  considerate, 

Ji4<tae  clear,  judge  overall).  The  variabilities  of  methods  A and  B were  quite 
similar. 

EASE  OF  USE.  Data  on  ease  of  use  came  from  the  opinion  questionnaire,  items 
1-3.  On  all  three  items  (work  load,  understanding,  interest)  there  were  no 
significant  or  important  differences  across  methods.  The  overall  means  were: 
work  load  - 2.0  ("about  right");  understanding  - 1.4  (between  "not  at  all  con- 
fused" and  "slightly  confused");  interest  - 2.8  ("slightly  interesting").  On 
the  item  "which  method  do  you  prefer,"  two  subjects  chose  A,  two  chose  B,  and 
two  chose  C.  Both  subjects  who  chose  A stated  that  the  option  to  type  in 
comments  was  a major  factor  in  their  choice. 

USEFULNESS  OF  RATINGS.  All  three  methods  provided  ratings  by  the  judge  of  the 
two  skills  and  overall  effectiveness.  Methods  B and  C also  provided  lists  of 
the  behaviors  observed.  All  of  this  should  be  useful  feedback  to  CCs.  Method 
B provided  a computer-processed,  time-weighted  average  score  for  each  of  the 
two  skills,  while  method  A provided  such  a score  for  overall  effectiveness. 
These  computer-processed  scores  would  serve  as  useful  checks  on  the  judges' 
opinions. 


TABLE 

1 . SCORE 

STANDARD  DEVIATIONS 

VARIABLE 

METHOD 

A 

METHOD 

B 

METHOD 

C 

Judge  considerate 

.55 

.51 

.81 

Judge  clear 

.21 

.33 

.87 

Judge  overall 

.24 

.23 

.93 

Computer  considerate 

- 

.57 

- 

Computer  clear 

- 

.51 

' - 

Computer  overall 

.21 

- 

- 
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SECTION  IV 


FOLLOW-ON  STUDY 


NEW  RATING  CONDITION 


Based  on  the  results  from  the  preliminary  study,  a new  rating  method  was 
designed.  It  was  most  similar  to  method  B,  in  that  skills  were  rated  first, 
followed  by  selection  of  behaviors.  The  method  of  selection  of  behaviors  first 
(method  C)  was  rejected  because  of  its  low  reliability.  Method  A was  not 
favored  because  it  did  not  yield  the  behavior  listings,  which  were  considered 
to  be  useful.  However,  the  comment  option  of  method  A was  retained. 

In  the  new  method,  the  judge  first  viewed  the  segment  without  making 
responses.  The  tape  was  then  reshown,  and  the  ji'dge  pressed  a key  on  the 
terminal  every  time  he/she  saw  an  instance  of  a considerate  behavior.  This 
keypress  stopped  the  tape.  The  judge  gave  a rating  (1-5)  to  indicate  the  level 
of  considerate  behavior  at  this  point  on  the  tape.  Following  this,  the  judge 
selected  the  considerate  behaviors  observed.  At  this  time,  the  judge  was  also 
given  the  opportunity  to  type  in  any  additional  comments.  The  tape  then  re- 
started, and  the  same  process  continued. 

At  the  end  of  the  segment,  the  judge  entered  an  overall  rating  for  con- 
siderate (1-5).  Then  the  tape  was  reshowi , and  the  same  procedure  was  followed 
for  the  skill  clear.  Finally,  the  judge  entered  an  overall,  global  rating. 
Sample  outputs  for  this  new  method  are  included  in  Appendix  D. 

PROCEDURE 

The  subjects  were  three  students  (one  female,  two  male)  from  Florida  Tech- 
nological University.  All  of  these  students  were  psychology  majors,  and 
familiar  with  behavioral  rating  techniques.  A fourth  subject  participated, 
but  experienced  considerable  difficulty  with  the  experiment,  so  the  data  was 
not  used  (the  subject  was  foreign-born  and  had  some  trouble  with  English).  The 
tapes  used  were  the  same  as  in  the  preliminary  study.  Each  subject  rated  all 
three  tape  segments  using  the  new  method.  After  rating  the  tapes,  each  subject 
filled  out  the  first  part  of  the  opinion  questionnaire  ("questions  for  each 
method" ) . 


RESULTS 


RELIABILITY.  To  assess  the  reliability  of  this  method,  the  intraclass  correla- 

23 

tion  coefficient  (as  described  by  Guilford  ) was  used.  This  is  a measure  of 
the  degree  to  which  different  raters  agree  on  the  rating  of  a given  variable, 
across  a number  of  ratees.  Table  2 presents  the  intraclass  correlation  co- 
efficients for  all  variables,  along  with  the  ratings. 

The  reliabilities  of  the  two  considerate  scores  were  low,  with  the  com- 
puter-averaged score  showing  a higher  reliability  than  the  judges'  score  (in- 
traclass correlation  coefficients  in  excess  of  .90  are  not  uncontnon  in  this 
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TABLE  2.  RATINGS  AND  INTRACLASS  CORRELATION  COEFFICIENTS 
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type  of  application).  The  reliabilities  of  the  clear  scores  were  moderate, 
and  equal  across  the  two  types.  The  reliability  of  the  overall  score  fell 
between  those  of  the  clear  and  considerate  scores. 

There  are  three  factors  which  might  have  contributed  to  the  unimpressive 
reliabilities.  First,  the  raters  were  given  little  training  and  practice  in 
the  use  of  the  method.  It  is  reasonable  to  assume  that  more  training  in  the 
use  of  the  method  and  the  meaning  of  the  variables  would  improve  the  reliabil- 
ity of  the  ratings.  This  is  especially  true  of  considerate,  which  has  a less 
obvious  meaning  than  clear. 

A second  factor  contributing  to  low  reliability  is  the  lack  of  variability 
in  the  performance  of  the  CCs  who  were  taped.  All  of  the  CCs  were  attempting 
to  do  their  best,  and  they  naturally  avoided  any  obviously  unclear  or  incon- 
siderate behaviors.  No  attempt  was  made  to  role  play  a "bad"  CC.  The  inclu- 
sion of  a clearly  bad  performance  would  have  increased  the  variance  of  both 
variables,  and  almost  certainly  would  have  improved  reliability. 

Finally,  intrinsic  characteristics  of  the  rating  variable  (such  as  the 
ease  of  applying  it)  would  affect  its  reliability.  The  emphasis  in  the  current 
study  was  on  how  to  facilitate  the  process  of  judging  and  providing  feedback  on 
interpersonal  performance,  regardless  of  the  particular  aspect  of  performance 
that  is  being  judged  or  fed  back.  Thus,  relatively  minor  consideration  was 
given  to  the  basic  reliability  of  the  rating  variables  in  this  context.  The 
present  techniques  can,  at  best,  allow  the  maximum  reliabilities  inherent  to  a 
rating  variable  to  be  manifested  in  the  ratings.  If  these  inherent  reliabil- 
ities are  low,  the  manifest  reliabilities  will  also  be  low,  no  matter  how  the 
ratings  are  accomplished.  For  example,  the  computer  time-weighted  method  con- 
tributed considerably  to  the  reliability  of  the  considerate  variable  (the  co- 
efficient increased  from  .26  to  .45).  Thus,  the  computer  method  is  considered 
to  be  of  benefit  to  ratings  of  considerateness  in  spite  of  the  relatively  low 
reliability  of  those  ratings.  Studies  are  needed  to  determine  rating  variable 
characteristics  that  are  conducive  to  good  reliability. 

EASE  OF  USE.  These  results  come  from  the  opinion  questionnaire.  On  the  "work 
load"  item,  all  subjects  reported  that  the  work  load  was  "about  right."  On  the 
"interest"  item,  two  subjects  selected  "somewhat  interesting"  and  one  selected 
"very  interesting."  On  the  "understanding"  item,  all  subjects  reported  that 
they  were  "not  at  all  confused."  Therefore,  it  was  concluded  that  this  method 
was  quite  acceptable  as  regards  ease  of  use. 

USFFULNESS  OF  RATINGS.  This  method  included  all  important  features  of  the 
three  preliminary  methods.  It  yielded  a judge's  average  for  each  variable,  a 
computer  average  for  each  variable,  an  overall  judge's  average,  a list  of  be- 
haviors observed,  and  judge's  comments.  All  three  subjects  freely  used  the 
comment  option.  All  ratings  and  coiments  were  keyed  to  the  points  in  the  tape 
where  the  relevant  behavior  occurred  as  well  as  an  overall  summary  at  the  end 
of  the  tape  recording.  The  availability  of  hard-copy  printouts  of  all  feedback 
information  would  enable  the  student  to  compare  his  performance  across  per- 
formance sessions. 

It  Is  therefore  concluded  that  this  method  would  be  very  useful,  both  for 
feedback  to  CCs  and  managerial  purposes. 
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SECTION  V 


DISCUSSION 

Previous  quantitative  research  on  the  use  of  videotape  as  a feedback  and 
evaluation  tool  in  the  interpersonal  skills  area  has  not  generally  yielded  en- 
couraging results.  A possible  cause  of  this  lack  of  positive  results  is  the 
failure  to  develop  sophisticated  and/or  standardized  methods  for  applying 
videotape  in  this  area.  The  current  research  attempted  to  take  a step  in  that 
direction  by  developing  a technique  for  rating  interpersonal  behavior  as  seen 
on  videotape,  with  the  aid  of  a computer. 

The  technique  designed  in  this  study  provides  detailed  information  on 
i nterpersonal  performance  which  should  be  highly  useful  in  a variety  of  ways. 
The  most  obvious  use  is  that  of  providing  performance  feedback  to  the  subjects 
of  the  tape.  This  technique  appears  to  be  simple  to  use,  and  should  not  re- 
quire extensive  rater  training.  One  questionable  area  is  that  of  reliability, 
and  it  is  likely  that  further  development  would  improve  the  reliability  of  this 
technique  substantially.  Further,  the  improved  reliability  of  the  considerate 
variable  as  a result  of  the  computer  technology  provides  encouraging  evidence 
that  additional  R&D  on  these  issues  can  contribute  importantly  to  the  area. 

The  contribution  of  the  computer  to  this  technique  was  a major  area  of 
concern.  If  desirable  judging  and  feedback  objectives  (as  represented  in  this 
study)  could  be  achieved  equally  well  (or  better)  without  computer  assistance, 
the  expense  of  the  computer  would  not  be  justified.  However,  after  considering 
the  probable  reduction  in  effectiveness  and  increment  in  costs  that  would  be 
associated  with  non-computer  administration  of  the  technique,  it  was  decided 
that  computer  assistance  is  cost-effective. 

The  cost  of  the  computer  equipment  used  in  this  study  was  approximately 
$30,000.  However,  this  equipment  performed  many  functions  in  addition  to  those 
related  to  this  study.  It  is  estimated  that  a minimum  system  capable  of  ad- 
ministering the  technique  described  here  would  cost  less  than  $2,000. 

Using  a non-computer  technique,  the  rater  would  proceed  in  much  the  same 
manner  as  with  the  computer,  except  that  he/she  would  stop  and  start  the  tape 
player  manually,  and  record  responses  on  paper  instead  of  typing  them  on  a key- 
board. The  major  additional  task  would  be  to  record  the  exact  point  on  the 
tape  where  the  responses  were  made.  This  would  require  the  rater  to  consult  a 
timer  or  counter  at  each  point  where  a response  was  made,  and  record  the 
reading.  At  the  end  of  the  rating  process,  the  time-weighted  averages  would 
have  to  be  manually  calculated. 

This  procedure  is  more  difficult  to  accomplish  than  it  may  first  appear. 
The  major  source  of  difficulty  is  that  the  judge  is  burdened  wi  th  a host  of 
"housekeeping"  tasks  (i.e.,  stopping  the  tape  at  the  appropriate  points, 
recording  times,  calculating  scores)  at  the  same  general  time  as  when  the  taped 
performance  is  to  be  rated.  The  process  of  rating  interpersonal  performance 
requires  intense  concentration  such  that  even  minor  additional  duties  can  be 
seriously  disruptive. 
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Due  to  the  early  stage  of  development  of  the  computer  programs  arid  the 
lack  of  definition  of  other  means  for  performing  the  tasks,  no  attempt  was 
made  to  rigorously  compare  the  various  computer  capabilities  with  other,  non- 
computer methods.  Anecdotal  evidence  on  this  point,  however,  was  obtained 
during  developmental  testing  of  the  programs.  In  these  tests,  the  present 
authors  performed  various  rating  and  feedback  tasks  manually  in  order  to  eval- 
uate the  appropriateness  of  candidate  programs  for  computer  implementation. 

It  became  quite  apparent  from  these  efforts  that  some  tasks  could  not  be  per- 
formed at  all  manually  with  only  one  operator,  and  some  of  these  tasks  could 
not  be  performed  satisfactorily  even  with  two  operators.  These  tasks  typically 
required  so  much  time  and  concentration  to  perform  that  the  operator  continual- 
ly lost  his  main  train  of  thought  and  had  to  repeat  previous  operations  to 
regain  it. 

The  operations  which  were  most  disruptive  to  the  total  task  were  the  com- 
putation of  rating  scores  and  the  control  of  videotape  presentation.  Also, 
these  operations  were  never  performed  as  accurately  manually  as  they  were  by 
computer  control  and  their  accomplishment  was  associated  with  frustration  and 
tedium.  These  problems  were  largely  resolved  through  use  of  computer  support. 
Thus,  the  computer  programs  increased  the  likelihood  that  the  operators  not 
only  would  be  able  to  properly  perform  the  tasks,  but  also  that  they  would  be 
willing  and  motivated  to  perform  them  properly. 

The  greatest  advantages  of  computer  assistance  in  the  current  application, 
however,  become  apparent  when  one  considers  the  process  of  giving  feedback  to 
a person  on  his/her  rated  performance.  Although  the  feedback  mode  was  not 
implemented  during  this  study,  the  provision  of  feedback  is  the  principal 
justification  for  developing  the  rating  procedure.  With  computer  assistance, 
the  ratee  would  watch  his/her  performance  on  tape,  and  the  computer  would  auto- 
matically stop  at  each  point  where  the  rater  had  made  a response.  The  ratee 
would  then  be  presented  with  a rating  on  the  skill  in  question,  a list  of  be- 
haviors noted  by  the  judge  as  relating  to  that  skill,  and  any  other  rater 
comments. 

Without  computer  control,  the  ratee's  attention  to  the  tape  and  the  feed- 
back would  be  seriously  disrupted  by  the  need  to  attend  to  the  manual  operation 
of  the  tape  player  and  timer,  and  coordination  of  the  delivery  of  the  feedback. 
If  another  person  were  to  operate  the  tape  player,  twice  the  man-hours  would  be 
required  for  the  feedback  process,  and  the  added  expense  would  soon  outweigh 
the  cost  of  computer  control  in  any  operational  situation.  There  is  also  a 
privacy  factor  to  consider,  since  the  ratee  might  be  more  ill  at  ease  with 
another  person  present. 

Thus,  the  system  performed  most  all  of  the  important  feedback  functions 
which  would  be  performed  by  a skillful  human  Instructor,  and  at  a lower  cost. 

In  addition,  the  computer  system  would  perform  these  functions  more  accurately, 
reliably,  objectively,  frequently,  patiently,  comprehensively,  and  with  greater 
potential  for  modification  and  Improvement  than  could  be  reasonably  expected 
from  a human  instructor.  Thus,  because  human  instructors  cannot,  will  not  and/ 
or  do  not  properly  perform  these  feedback  functions,  the  present  system  would 
provide  feedback  that  is  far  superior  to  that  typically  provided  in  inter- 
personal skills  training  programs. 
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The  current  computer  capabilities,  and  additional  capabilities  suggested 
by  this  study,  could  provide  the  means  for  ’mpl ementing  a variety  of  other 
techniques  for  aiding  the  judging  process  that  would  go  well  beyond  the  com- 
plexity of  this  preliminary  demonstration.  The  computer  capabilities  demon- 
strated or  suggested  in  this  study  provide  flexibility  to  explore  potentially 
useful  judging  and  feedback  techniques  which  would  be  impossible  to  accomplish 
by  other  means. 

Additional  functions,  not  implemented  in  the  current  study,  but  which  the 
computer  could  readily  perform,  involve  the  automatic  storage  and  processing  of 
data  on  the  performance  of  ratees.  For  example,  a ratee  could  be  given  feed- 
back on  how  his  performance  compares  with  all  other  persons  rated  with  the 
system.  Alternately,  the  ratings  of  several  raters  could  be  combined  to  pro- 
vide feedback  from  different  sources.  Or,  the  percent  of  time  that  a ratee 
exhibited  a given  characteristic  could  be  computed.  Another  function  which 
was  briefly  mentioned  is  random  access  to  tape  positions.  The  tape  player 
could  be  made  to  send  position  information  to  the  computer,  which  would  allow 
the  computer  to  locate  any  point  on  the  tape  automatically  (using  rewind  and 
fast  forward  modes,  as  well  as  play  mode).  As  an  example  of  the  use  of  this 
function,  a ratee  could  choose  to  review  only  those  portions  of  a tape  wiiere 
he/she  was  rated  as  being  inconsiderate.  The  computer  could  control  the  player 
such  that  only  these  segments  were  shown,  fast  forwarding  between  them  so  that 
intervening  material  could  be  skipped. 

In  conclusion,  it  seems  apparent  that  further  research  on  the  videotape 
rating  technique  developed  in  this  study  is  warranted,  and  that  the  use  of  a 
computer  to  assist  in  this  technique  is  effective  and  efficient. 
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APPENDIX  A 

INSTRUCTIONS  TO  RATERS 


INTRODUCTION 

In  this  study,  you  will  be  rating  the  performance  of  persons  in  inter- 
personal situations  (as  seen  on  videotape).  The  persons  you  will  be  rating 
are  Navy  Company  Comnanders.  It  is  their  job  to  lead  companies  of  80  new  re- 
cruits through  eight  weeks  of  basic  training.  The  sections  of  tape  which  you 
will  see  involve  interactions  between  Company  Commanders  (CCs)  and  their  re- 
cruits. Some  of  these  interactions  are  role-played  and  some  are  taken  from 
real-life  situations. 
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VARIABLES 

The  variables  on  which  you  will  be  rating  the  CCs 
fall  into  three  levels: 

1.  OVERALL  EFFECTIVENESS  LEVEL  - How  effective  is  the  CC  in  interacting  with 
the  recruit? 

2.  SKILL  LEVEL  - How  well  does  the  CC  perform  certain  skills?  The  skills  you 
will  be  rating  are: 

CONSIDERATE  - To  what  extent  does  the  CC  treat  the  recruit  like  a human 
being? 

CLEAR  - How  clear  is  the  CC's  communication  with  the  recruit? 

3.  BEHAVIOR  LEVEL  - How  well  is  the  CC  performing  certain  behaviors?  The 
behaviors  which  you  will  be  looking  for  fall  under  the  two  skills  above: 

CONSIDERATE  - POSITIVE 

1.  The  CC  asked  the  recruit  how  he/she  felt  about  the  situation. 

2.  The  CC  expressed  confidence  in  the  recruit's  abilities. 

3.  The  CC  was  patient  with  the  recruit. 

4.  The  CC  used  the  recruit's  name. 

CONSIDERATE  - NEGATIVE 

1.  The  CC  showed  disinterest  in  the  recruit's  feelings. 

2.  The  CC  belittled  the  recruits  abilities. 

3.  The  CC  was  impatient  with  the  recruit. 

4.  The  CC  used  abusive  names. 

CLEAR  - POSITIVE 

1.  The  CC  gave  specific  details. 

2.  The  CC  encouraged  recruits  to  ask  questions. 

3.  The  CC  asked  questions  of  recruits  to  see  if  they  understood. 

4.  The  CC  gave  reasons  to  show  the  relevance  of  what  he/she  was  saying. 

CLEAR  - NEGATIVE 

1.  The  CC  was  vague. 

2.  The  CC  discouraged  questions. 

3.  The  CC  failed  to  question  the  recruits  for  their  understanding. 

4.  The  CC  failed  to  give  reasons  although  the  situation  called  for  it. 

Please  study  these  variables  so  that  you  will  be  familiar  with  them  as  you 
rate  the  tapes.  Keep  this  sheet  handy  to  refer  to. 


NAVTRAEQU I PC EN  IH-308 
RATING  METHOD  A 

1.  You  will  see  one  of  the  segments  which  you  viewed  previously,  in  v^ii ch  one 
CC  interacts  with  two  different  recruits.  As  you  watch,  you  will  make  a 
continual  rating  of  the  OVERALL  EFFECTIVENESS  of  the  CC's  performance. 

You  will  do  this  by  pressing  keys  1 - 5 on  the  terminal* 

Key  1 - the  CC  is  totally  ineffective 

Key  2 - the  CC  is  somewhat  ineffective 

Key  3 - the  CC  is  neither  effective  nor  ineffective 

Key  4 - the  CC  is  somewhat  effective 

Key  5 - the  CC  is  very  effective 

For  example,  if  you  feel  that  the  CC  is  "somewhat  effective"  at  a given 
point  in  the  tape,  you  will  press  4.  The  computer  will  continue  to  record 
a rating  of  4 until  you  press  another  key.  You  should  change  your  key 
selection  every  time  your  opinion  changes.  The  tape  will  start  with  key  3 
in  effect.  Make  your  first  response  when  you  have  enough  information  to 
decide  how  effectively  the  CC  is  performing.  Do  not  press  any  keys  while 
the  recruit  is  speaking. 

2.  You  will  watch  the  segment  again.  This  time  the  tape  will  stop  at  each 
point  where  you  changed  your  rating  in  the  previous  viewing.  When  it 
stops,  you  will  be  asked  if  your  change  in  rating  was  caused  by  the  CC's 
performance  in  the  CLEAR  or  CONSIDERATE  area.  You  will  also  have  an 
opportunity  to  type  in  a comnent  on  the  CC's  performance. 

3.  After  the  second  showing,  the  computer  will  display  the  number  of  times 
you  saw  each  skill.  You  will  then  give  the  CC  a suninary  rating  on  each 
skill,  as  well  as  an  overall  rating. 
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NAVTRAEQU IPCEN  IH-308 
RATING  METHOD  B 

1.  You  will  see  one  of  the  segments  which  you  viewed  previously.  As  you 

watch,  you  will  make  a continual  rating  of  the  CC's  performance  on  one^  of 
the  two  skills.  You  will  do  this  by  pressing  keys  1 - 5 on  the  terminal . 

Key  1 - the  CC  is  performing  very  poorly  on  the  skill 

Key  2 - the  CC  is  performing  poorly  on  the  skill 

Key  3 - the  CC  is  performing  at  an  average  level  on  the  skill 

Key  4 - the  CC  is  performing  well  on  the  skill 

Key  5 - the  CC  is  performing  very  well  on  the  skill 

For  example,  if  you  see  that  the  CC  is  performing  well  on  the  skill  at  a 
certain  point  on  the  tape,  you  will  press  4.  The  computer  will  continue 
to  record  a rating  of  4 until  you  press  another  key.  You  should  change 
your  key  selection  every  time  your  opinion  changes.  The  tape  will  start 
with  key  3 in  effect.  Make  your  first  response  vrfien  you  have  enough  in- 
formation to  decide  how  effectively  the  CC  is  performing.  Do  not  press 
any  keys  while  the  recruit  is  speaking. 

2.  You  will  watch  the  segment  again.  This  time,  the  tape  will  stop  at  each 

point  where  you  changed  your  rating  in  the  previous  viewing.  When  it 
stops,  you  will  select  the  behaviors  which  caused  you  to  change  your  rating. 

3.  At  the  end  of  the  second  viewing,  the  total  number  of  observations  of  each 
behavior  will  be  displayed.  You  will  then  give  the  CC  an  overall  rating 
on  the  skill. 

4.  Steps  1 - 3 will  be  repeated  for  the  second  skill. 

5.  You  will  give  the  CC  an  overall  effectiveness  rating. 
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NAVTRAEQU IPCEN  IH-308 
RATING  METHOD  C 

1.  You  will  see  one  of  the  segments  which  you  viewed  previously.  As  you 
watch,  you  will  press  the  space  bar  every  time  you  notice  one  of  the  be- 
haviors being  performed.  Keep  the  list  of  behaviors  handy  so  that  you  can 
refer  to  it.  The  tape  will  not  stop  during  this  run-through. 

2.  You  will  watch  the  segment  again.  This  time  the  tape  will  stop  at  each 
point  where  you  pressed  the  space  bar.  When  it  stops,  you  will  select 
the  behaviors  which  you  observed. 

3.  At  the  end  of  the  second  viewing,  the  total  number  of  observations  of  each 
behavior  will  be  displayed.  You  will  then  give  the  CC  an  overall  rating 
for  each  skill. 

4.  You  will  give  the  CC  an  overall  effectiveness  rating. 
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NAVTRAEQUIPCEN  IH-308 
APPENDIX  B 


ROLE-PLAY  SCENARIOS 


ROLE-PLAYING  EXERCISE 
COMPANY  COMMANDER 

It  is  the  4-2  day  of  training.  Recruit  Wilson  has  been  having  serious 
problems  throughout  training,  mainly  with  MED  inspections.  It  has  been  decided 
that  Wilson  will  go  to  T.U.  4013  (fold  and  stow).  Wilson  will  go  to  the 
Division  office  at  0800  tomorrow;  then  will  return  to  the  barracks,  pack  his 
(her)  gear,  and  go  to  T.U.  4013.  Generally,  recruits  stay  in  T.U.  4013  for 
two  to  ten  days,  then  go  to  a T.U.  which  is  at  the  day  of  training  vrtiere  they 
left  their  original  T.U. 

Recruit  Wilson  has  never  been  a disciplinary  problem.  Wilson  has  always 
tried  hard,  but  seems  a little  "slow"  and  has  been  overwhelmed  by  all  of  the 
things  that  need  to  be  learned  at  RTC.  It  is  likely  that,  with  the  extra 
training  in  T.U.  4013,  Wilson  will  be  able  to  make  it  through  RTC. 

Now  you  need  to  tell  Wilson  what  has  been  decided.  Think  about  what  you 
want  to  say;  and  when  you  are  ready,  begin. 

o 

ROLE-PLAYING  EXERCISE 
RECRUIT 

You  have  just  been  called  to  the  CC's  office.  You  expect  that  this  has 
something  to  do  with  your  poor  performance  on  inspections.  You  are  very 
worried  that  you  may  be  kicked  out  of  the  Navy,  because  you  would  really  like 
to  stay  in  and  make  a career  of  it. 

Figure  B-l . Role-Play  #1 
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NAVTRAEQU I PC EN  IH-308 
ROLE-PLAYING  EXERCISE 


RCPO 

You  are  an  RCPO  at  RTC  Orlando.  CPO  Allen  (substitute  name  of  person 
playing  CC  role)  has  been  the  CC  in  your  company  since  its  formation  four 
weeks  ago. 

Everything  seemed  to  be  going  along  pretty  well  for  the  first  couple  weeks 
of  training.  For  the  past  two  weeks,  however,  the  company  has  been  falling 
apart  and  the  CC  won't  do  anything  to  stop  it. 

The  company  scores  on  the  last  inspection  (i.e.,  8 discrepancies  in 
personnel,  13  in  locker)  were  way  down  from  the  previous  inspections  (4  in 
personnel,  7 in  locker);  there  has  been  an  increase  in  street  marks  in  the 
last  week;  and  the  DPO  had  to  establish  order  in  the  barracks  twice  in  the 
last  two  weeks. 

You  feel  that  these  problems  are  due  mainly  to  the  attitude  and  perform- 
ance of  the  CC.  Recently,  the  CC  is  expecting  you  to  do  a lot  of  things  that 
seem  to  be  his  job.  You  don't  think  it's  fair  of  him  to  expect  you  to  run  the 
company  practically  on  your  own.  Many  problems  are  arising  that  you  don't  have 
the  time  or  ability  to  handle.  You're  not  sure  what  is  expected  of  you  and  the 
other  RPOs  feel  the  same  way. 

Many  of  the  recruits  also  are  complaining  about  the  management  of  the 
company.  Most  recruits  agree  that:  (a  ) the  CC  should  be  around  more  to  watch 
over  things;  (b)  he  is  not  interested  in  the  company  anymore;  (c)  he  is  con- 
cerned only  about  winning  flags  with  the  least  possible  effort  from  him;  (d) 
he  has  little  confidence  in  their  ability  to  perform  well;  and  (e)  much  of 
recruit  training  is  a waste  of  time. 

When  you  have  tried  to  tell  the  CC  about  these  problems,  he  becomes  angry, 
throws  the  blame  on  you,  and  shows  no  interest  in  changing  his  ways.  It  is 
quite  upsetting  to  be  yelled  at  for  problems  that  are  the  CC's  fault,  especial- 
ly when  you  have  been  killing  yourself  to  run  the  company  and  the  CC  will  take 
no  responsibility  for  the  problems. 

You  have  decided  that  maybe  it's  better  not  to  talk  to  the  CC  about  these 
problans.  You  tune  him  out  when  he  blames  you  for  the  problems.  Until  he 
shows  that  he  will  meet  the  problems  more  reasonably,  you  decided  to  play  down 
your  discussions  with  him.  Instead  of  talking  with  him  about  the  problems, 
you  are  sending  as  many  of  the  problem  cases  as  you  can  to  the  CC.  This  way, 
he  will  be  able  to  see  first  hand  what  you're  up  against  and  it  will  force  him 
to  become  more  involved  with  the  company  again. 

However,  you  would  like  to  be  able  to  discuss  these  matters  with  the  CC. 
Thus,  If  (and  only  if)  he  shows  signs  that  you  can  trust  him  to  be  more  reason- 
able and  considerate  will  you  tell  him  what  you  think. 

Figure  B-2.  Role-Play  #2 


31 


NAVTRAEQU I PCEN  IH-308 

ROLE-PLAYING  EXERCISE 
CC 

You  are  a CC  at  RTC,  Orlando.  Bob  Sanford  (substitute  name  of  person 
playing  RCPO)  has  been  the  RCPO  in  your  company  since  its  formation  four  weeks 
ago. 

RCPO  Sanford  seemed  to  be  doing  a satisfactory  job  for  the  first  three 
weeks  of  training.  However,  there  has  been  some  evidence  in  the  past  week  to 
suggest  that  the  RCPO  is  not  doing  an  adequate  job.  This  evidence  was  obtained 
from  the  performance  of  the  company  and  observations  of  and  conversations  with 
the  RCPO  and  other  members  of  the  company. 

The  company  scores  on  the  last  inspection  (i.e.,  8 discrepancies  in 
personnel,  13  in  locker)  were  way  down  from  the  previous  inspections  (4  in 
personnel,  7 in  locker);  there  has  been  an  increase  in  street  marks  in  the 
last  week;  and  the  DPO  had  to  establish  order  in  the  barracks  twice  in  the 
last  two  weeks. 

You  feel  that  these  problems  are  due  mainly  to  the  attitudes  and  perform- 
ance of  the  RCPO.  The  problems  began  soon  after  you  decided  to  stop  "baby- 
sitting" for  the  recruits  and  to  allow  them  to  take  on  more  of  the  responsibil- 
ity for  running  themselves.  The  RCPO  seems  to  be  reluctant  to  take  on  the 
added  responsibility.  He  is  sending  many  problems  to  you  that  he  and  the  other 
recruits  ought  to  be  resolving  for  themselves.  You  have  been  sending  these 
problems  right  back  to  the  RCPO.  In  fact,  you  might  have  been  sending  back 
more  than  usual  to  the  RCPO  just  to  give  him  the  idea  that  he  is  not  going  to 
get  out  of  work  by  sending  it  to  you. 

The  thing  that  upsets  you  most  is  that  the  RCPO  appears  to  be  creating 
poor  attitudes  in  the  rest  of  the  company  with  his  complaints  about  you. 

You  have  noticed  that  something  is  disturbing  the  RCPO  in  his  recent 

interactions  with  you.  Although  you  are  not  sure  exactly  what  it  is,  you  feel 
that  either  he  is  ashamed  of  his  poor  showing  recently  as  an  RCPO  or  he  might 
have  personal  problems.  He  is  quieter  with  you  than  usual  and  he  becomes  emo- 
tionally upset  when  you  try  to  instruct  him  on  how  to  run  the  company.  When  he 
is  not  upset,  he  appears  to  be  bored  and  unconcerned  about  what  you  say.  Some- 
times he  even  claims  you  never  told  him  some  things  when  you  know  you  have. 

You  don't  want  to  return  to  "babysitting"  for  the  company.  You  have  de- 
cided to  make  one  more  effort  to  straighten  out  the  RCPO,  before  you  replace 
him.  You  don't  want  to  replace  him  because  he  has  good  potential  and  also  be- 
cause it  might  demoralize  the  company,  but  you  will  if  he  doesn't  come  around 
and  show  a better  attitude  in  your  next  discussion  with  him.  You  want  to  do 
all  you  can  in  this  meeting  that  will  increase  your  confidence  that  the  RCPO 
will  start  taking  some  positive  actions  to  improve  the  situation  and  at  the 

same  time  improve  the  working  relationship  between  you  and  him.  If  all  your 

efforts  in  this  meeting  do  not  leave  you  confident  that  the  difficulties  in  the 
company  will  improve,  you  have  decided  to  relieve  the  RCPO  on  the  spot,  and 
you  will  proceed  to  do  so. 

Figure  B-2.  Role-Play  #2  (Contd) 
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NAVTRAEQU IPCEN  IH-308 
Behavioral  Checklist 


Concrete:  + gave  details  of  what  RCPO's  job  consists  of 

+ gave  details  in  feedback  about  how  well  RCPO  is  doing  his  job 

+ gave  details  in  instruction  about  what  the  particular  problems 
are  and  how  the  RCPO  should  handle  them: 


(Problems  are:  (a)  RCPO  sending  CC  too  many  problems;  (b)  low 
inspection  scores;  (c)  RCPO  talking  against  CC;  (d)  TU  has 
poor  attitudes;  (e)  DPO  needed  to  establish  order;  (f)  RCPO 
not  his  usual  self  (quieter,  upset,  unmotivated)  (g)  communi- 
cation problem  (with  RCPO).) 

Relevant:  + gave  good  reasons  for  allowing  the  TU  to  run  itself  and  gave 
good  reasons  for  the  benefits  of  recruit  training  in  general 

+ told  how  running  the  TU  will  benefit  the  RCPO 

Human:  + admitted  that  he  might  not  have  given  enough  help  with  the 

company 


+ showed  that  he  is  interested  in  the  recruits'  welfare 

+ told  how  he  feels  about  the  current  problems  with  the  company, 
RTC,  etc. 

+ admitted  that  he  does  not  know  enough  about  the  problems  to  offer 
the  best  help 


+ put  the  RCPO  at  ease  with  some  humor,  expressions  of  personal 
experiences,  etc. 


+ showed  he  is  interested  in  RCPO's  welfare 


+ used  words  "we",  "us",  etc. 


Considerate: 


+ expressed  confidence  in  RCPO's  ability  to  do  the  job 


+ asked  RCPO  how  he  and  others  feel  about  him,  RTC,  problems  in 
the  company,  etc. 

+ did  not  become  unduel y angry  or  yell  at  RCPO 

+ told  RCPO  exactly  what  he  planned  to  do  regarding  relieving 
him  and  how  he  thinks  the  problems  are  due  to  the  RCPO's 
performance 


Figure  B-2.  Role-Play  #2  (Contd) 
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Set  Expectations:  + told  RCPO  the  kinds  of  problems  that  he  will  host  likely 
* encounter  at  this  stage  of  training 

Timely:  + told  RCPO  what  are  the  most  important  things  to  do  and  what  may  be 
less  Important 

+ repeated  important  information  often,  summarized  it  well,  or  other- 
wise made  sure  RCPO  would  remember  it 

Aware:  + observed,  questioned,  and  probed  the  RCPO  for  his  understanding  of 

what  was  said 

Figure  B-2 . Role-Play  #2  (Contd) 


NAVTRAEQU IPCEN  IH-308 
ROLE-PLAYING  EXERCISE 


RECRUIT 


You  are  recruit  Bradford  in  TU  150  which  is  in  the  2-4  day  of  training. 
Recently,  you  have  been  having  some  trouble  relating  to  your  section  leader, 
RPO  Sharkey.  But  this  morning  he  jumped  all  over  you  for  losing  a point  in 
the  last  inspection.  When  you  told  him  that  he  showed  you  the  wrong  way  to 
fold  his  shirt  that  he  lost  the  point  on,  he  became  extremely  offensive  and 
started  shouting  and  swearing  about  how  he  is  going  to  "take  care  of  you"  if 
you  try  to  blame  him  for  your  mistakes.  You  felt  this  was  a physical  threat 
that  you  didn't  have  to  take,  so  you  went  to  RCPO  Cook  to  tell  him  about  it. 
Cook  right  away  started  to  plan  out  what  to  say  to  CC  Altman  about  the  inci- 
dent, which  bothered  you  because  you  didn't  want  to  make  such  a big  thing  out 
of  it.  But  since  RCPO  Cook  was  so  much  in  favor  of  it,  you  felt  you  had  to  go 
along  with  his  plan. 

So  you  and  RCPO  Cook  got  your  story  together  (allow  time  for  this)  and 
went  to  see  CC  Altman. 

You  are  just  starting  to  discuss  this  matter  with  CC  Altman  as  the  role 
play  begins. 


CC 

You  are  CC  Altman  in  TU  150.  Today,  on  the  2-4  day  of  training,  you 
were  approached  by  recruit  Bradford  and  RCPO  Cook  from  your  company.  Bradford 
reported  that  his  section  leader,  Sharkey,  maltreated  him.  Up  to  now,  RPO 
Sharkey  seemed  to  be  doing  fine.  However,  you  are  concerned  about  personal 
conflicts  that  you  have  noticed  among  recruits  that  you  feel  are  decreasing 
motivation  In  the  Training  Unit  and  probably  were  responsible  for  the  poor 
showing  on  the  last  inspection.  You  told  the  RCPO  to  watch  for  these  conflicts 
and  to  try  to  improve  the  team  spirit  of  the  company.  You  think  that  this  in- 
cident is  an  example  of  these  conflicts,  so  you  are  eager  to  get  to  the  bottom 
of  it. 

You  are  just  starting  to  discuss  this  matter  with  Bradford  and  Cook  as 
the  role  pi  ay  begins. 


RPO 

You  are  RPO  Sharkey,  section  leader  in  TU  150.  You  woke  up  to  a particu 
larly  bad  day  today  --2-4  day  of  training.  You  were  just  out  of  bed  when 
RCPO  Cook  was  all  over  you  about  two  recruits  in  your  section  who  lost  points 
in  the  last  inspection.  He  said  you  better  get  il  together  or  you  won't  be 
section  leader  for  very  long. 

Figure  B-3.  Role-Play  #3 
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NAVTRAEQU I PCEN  IH-308 
RPO  (Contd) 


RCPO  Cook  has  been  out  to  get  you  since  the  company  formed  and  you  know 
he  is  just  looking  for  a chance  to  get  you  replaced.  This  is  especially  up- 
setting because  everyone  at  home  is  very  proud  that  you  were  chosen  section 
leader.  In  fact,  you  just  received  a letter  from  your  father,  who  is  very 
sick,  In  which  he  said  how  happy  it  would  make  him  to  be  able  to  see  you  grad- 
uate as  an  RPO.  You  have  been  telling  other  recruits  that  you  will  get  RCPO 
Cook  replaced  before  he  does  it  to  you.  Most  of  the  other  recruits  agree  with 
you  that  Cook  should  be  replaced,  but  say  he  has  the  CC  under  his  thumb  and 
that  you  don't  have  a chance.  You  feel  that  Cook  is  not  a good  RCPO  because  of 
the  abusive  manner  in  which  he  deals  with  people.  You  really  think  the  company 
would  be  a lot  better  off  without  RCPO  Cook. 

RCPO  Cook  bothered  you  so  much  that  morning  that  when  recruit  Bradford 
started  to  blame  you  for  his  mess-up  on  the  inspection,  you  blew  your  top  and 
yelled  at  him  a lot.  He  said  he  wasn't  going  to  take  this  from  you  and  went  \ 

to  see  RCPO  Cook  about  it. 

You  know  that  Cook  and  Bradford  went  to  the  CC's  office  to  talk  about  the 
matter.  You're  afraid  that  the  CC  won't  understand  that  you're  doing  as  well  < 

as  anyone  else  and  that  RCPO  Cook  is  just  using  this  incident  to  get  you. 

You're  getting  your  story  together  as  you  wait  to  hear  what  happens,  because 
you  know  this  could  be  a real  problem  for  you. 

(RPO  should  wait  in  another  room. ) 

Figure  B-3.  Role-Play  #3  (Contd) 
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NAVTRAEQUIPCEN  IH-308 
APPENDIX  C 


QUESTIONNAIRE 

The  process  of  evaluation  in  the  interpersonal  skills  area  involves  the 
performer  being  judged  and  informed  about  how  well  he/she  is  doing  on  the  job. 
The  general  purpose  of  this  study  is  to  determine  how  the  computer  might  help 
to  accomplish  some  of  these  evaluation  and  feedback  tasks.  Your  honest  and 
thoughtful  answers  to  the  following  questions  are  critical  to  this  study  and 
will  be  appreciated. 


Questions  for  Each  Method 

1.  I felt  that  the  work  load  in  this  method  was: 

a.  much  too  heavy 

b.  somewhat  too  heavy 

c.  slightly  too  heavy 

d.  about  right 

e.  too  light 

2.  Using  this  method,  I was: 

a.  very  confused  about  what  I was  supposed  to  be  doing 

b.  somewhat  confused  about  what  I was  supposed  to  be  doing 

c.  slightly  confused  about  what  I was  supposed  to  be  doing 

d.  not  at  all  confused  about  what  I was  supposed  to  be  doing 

3.  I found  this  method: 

a.  very  interesting 

b.  somewhat  interesting 

c.  slightly  interesting 

d.  boring 

4.  Compared  with  other  ways  for  describing  and  evaluating  interpersonal  per- 
formance that  I can  think  of,  this  method  was: 

a.  much  more  useful 

b.  more  useful 

c.  neither  more  nor  less  useful 

d.  less  useful 

e.  much  less  useful 

5.  Please  describe  any  modifications  to  this  method  which  you  think  might 
make  1 t more  useful : 


NAVTRAFQU I PC FN  IH-308 
Final  Questions 

1.  The  basic  idea  of  using  a computer  to  help  a person  evaluate  videotaped 
interpersonal  performance  is: 

a.  nonsense 

b.  probably  not  worth  pursuing 

c.  possibly  not  worth  pursuing 

d.  probably  worth  pursuing 

e.  definitely  worth  pursuing 

2.  Which  of  the  methods  did  you  prefer?  Why? 


3. 


4. 


What  did  you  like  most  and  least  about  each  method? 

A - most 
1 east 
B - most 
least 
C - most 
1 east 

Please  write  below  any  comments  which  might  help  us  in  this  study: 
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APPENDIX  D 

SAMPLE  COMPUTER  OUTPUTS 

FROM  TO  KEY 

0 MIN  0 SEX'  0 MIN  33  SEC  3 

***COMMENTS*** 

NONE 

***SKILLS  EXHIBITED  THIS  SEGMENT*** 

CLEAR 

FROM  TO  KEY 

0 MIN  33  SEC  0 MIN  48  SEC  4 

***COMMENTS*** 

THE  CC  THREATENED  THE  RECRUIT 
***SKILLS  EXHIBITED  THIS  SEGMENT*** 

CONSIDERATE 

FROM  TO  KEY 

0 MIN  48  SEC  3 MIN  50  SEC  2 

***COMMENTS*** 

NONE 

***SKILLS  EXHIBITED  THIS  SEGMENT*** 

CONSIDERATE  CLEAR 

FROM  TO  KEY 

3 MIN  50  SEC  5 MIN  00  SEC  4 

***COMMENTS*** 

EXPRESSED  A LOT  OF  CONCERN  FOR  THE  RECRUIT  AS  A PERSON 
AND  EXPRESSED  CONFIDENCE  IN  THE  RECRUITS  ABILITIES 

***SKILLS  EXHIBITED  THIS  SEGMENT*** 

CONSIDERATE  CLEAR 

Figure  D-l.  Method  "A"  Output 
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CALCULATED  VTR  EFFECTIVENESS  AVERAGE 3.550 

JUDGES  SELECTED  EFFECTIVENESS  AVERAGE 4.000 

SKILL  OBSERVATIONS  EFFECTIVENESS 

AVERAGE 

4.500 
4.000 

Figure  D-l.  Method  "A"  Output  (ContdJ 


CONSIDERATE  3 

CLEAR  3 
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EVALUATED  SKILL  IS  CONSIDERATE 

FROM  TO  KEY 

0  MIN  0 SEC  0 MIN  55  SEC  3 

***EtEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

7)  WAS  IMPATIENT  WITH  THE  RECRUIT 

FROM  TO  KEY 

0 MIN  55  SEC  1 MIN  38  SEC  4 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

9)  A BEHAVIOR  NOT  LISTED 

FROM  TO  KEY 

1 MIN  38  SEC  2 MIN  52  SEC  3 

*** BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

2)  EXPRESSED  CONFIDENCE  IN  THE  RECRUITS  ABILITIES 

FROM  TO  KEY 

2 MIN  52  SEC  5 MIN  00  SEC  4 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

:tl)  ENCOURAGED  THE  RECRUIT  TO  ASK  QUESTIONS 

EVALUATED  SKILL  IS  ■CONSIDERATE" 

OVERALL  SKILL  EFFECTIVENESS  AVERAGE ....  3 . 500 
CALCULATED  EFFECTIVENESS  AVERAGE 3.490 

Figure  D-2.  Method  "B"  Output 
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EVALUATED  SKILL  IS  CLEAR 

FROM  TO  KEY 

0 MIN  0 SEC  0 MIN  37  SEC  3 

♦ ♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENTS 
1 4 ) WAS  VAGUE 

FROM  TO  KEY 

0 MIN  37  SEC  2 MIN  15  SEC  4 

♦ ♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT^™ 

10)  GAVE  SPECIFIC  DETAILS 

FROM  TO  KEY 

2 MIN  15  SEC  3 MIN  44  SEC  2 

♦ ♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT^™ 

13)  GAVE  REASONS  FOR  THE  THINGS  RECRUITS  DO  AT  RTC 
18)  A BEHAVIOR  NOT  LISTED 

FROM  TO  KEY 

3 MIN  44  SEC  5 MIN  00  SEC  4 

♦ ♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT^™ 

.14)  WAS  VAGUE 

EVALUATED  SKILL  IS  ‘CLEAR* 

OVERALL  SKILL  EFFECTIVENESS  AVERAGE ....  4 . 000 
CALCULATED  EFFECTIVENESS  AVERAGE 3.400 

JUDGE'S  OVERALL  EFFECTIVENESS  AVERAGE 4.000 

Figure  D-2.  Method  "B"  Output  (Contd) 
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FROM  TO 

0  MIN  0 SEC  0 MIN  33  SEC 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

14  ) WAS  VAGUE 

9 ) A CONSIDERATE  BEHAVIOR  NOT  LISTED 

FROM  TO 

0 MIN  33  SEC  1 MIN  47  SEC 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

10)  GAVE  SPECIFIC  DETAILS 

18)  A CLEAR  BEHAVIOR  NOT  LISTED 

FROM  TO 

1 MIN  47  SEC  2 MIN  59  SEC 

♦♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

5)  SHOWED  DISINTEREST  IN. THE  RECRUIT'S  FEELINGS 

6)  BELITTLED  THE  RECRUIT'S  ABILITIES 

FROM  TO 

2 MIN  59  SEC  5 MIN  00  SEC 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

18)  A CLEAR  BEHAVIOR  NOT  LISTED 

JUDGE'S  OVERALL  EFFECTIVENESS  AVERAGE 2.500 

SKILL  AVERAGE 

CONSIDERATE  2.000 

CLEAR  3.000 

Figure  D-3.  Method  "C"  Output 
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EVALUATED  SKILL  IS  CLEAR 

FROM  TO  KEY 

0 MIN  0 SEC  1 MIN  51  SEC  4 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

10)  GAVE  SPECIFIC  DETAILS 

FROM  TO  KEY 

1 MIN  51  SEC  2 MIN  34  SEC  2 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

14 ) WAS  VAGUE 

16)  FAILED  TO  QUESTION  THE  RECRUITS  FOR  THEIR  UNDERSTANDING 
***BEHAVIOR  NOT  LISTED*** 

INCORRECTLY  INFERS  ATTITUDE  FROM  BEHAVIOR 

FROM  TO  KEY 

2 MIN  34  SEC  3 MIN  49  SEC  3 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

10)  GAVE  SPECIFIC  DETAILS 

13)  GAVE  REASONS  TO  SHOW  RELEVANCE  OF  WHAT  WAS  SAID 

FROM  TO  KEY 

3 MIN  49  SEC  5 MIN  00  SEC  3 

***BEHAVIORS  EXHIBITED  THIS  SEGMENT*** 

10)  GAVE  SPECIFIC  DETAILS 

13)  GAVE  REASONS  TO  SHOW  RELEVANCE  OF  WHAT  WAS  SAID 
***BEHAVIOR  NOT  LISTED*** 

NOT  CLEAR  IN  QUESTIONING 


SKILL  AVERAGE 

CLEAR  3.000 

CALCULATED  5 MINUTE  VIDEOTAPE  AVERAGE ....  2 . 940 


Figure  D-4.  New  Method  Output 
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EVALUATED  SKILL  IS  CONSIDERATE 

FROM  TO  KEY 

0 MIN  0 SEC  1 MIN  02  SEC  2 

♦ ♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT^^ 

7)  WAS  IMPATIENT  WITH  THE  RECRUIT 
4)  USED  THE  RECRUIT'S  NAME 

FROM  TO  KEY 

1 MIN  02  SEC  1 MIN  47  SEC  2 

♦♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT ♦♦♦ 

6 ) BELITTLED  THE  RECRUIT'S  ABILITIES 

FROM  TO  KEY 

1 MIN  47  SEC  3 MIN  33  SEC  3 

♦ ♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT^W 

:L0>  ASKED  THE  RECRUIT  HOW  HE/SHE  FELT  ABOUT  THE  SITUATION 

♦ ♦♦BEHAVIOR  NOT  LISTEDW^ 

SEEMED  DETACHED 

FROM  TO  KEY 

3 MIN  33  SEC  5 MIN  00  SEC  2 

♦♦♦BEHAVIORS  EXHIBITED  THIS  SEGMENT^W 

7)  WAS  IMPATIENT  WITH  THE  RECRUIT 

♦ ♦♦BEHAVIOR  NOT  LISTEDM^ 

POOR  LISTENING 


SKILL  AVERAGE 

CONSIDERATE  2.000 

CALCULATED  5 MINUTE  VIDEOTAPE  AVERAGE 2.120 

JUDGE'S  OVERALL  EFFECTIVENESS  AVERAGE 2.700 


Figure  0*4.  New  Method  Output  (Cor)td) 
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